MDM technology (Master Data Management) is used to create a unified and consistent view of enterprise-wide data and is used to make data available to all external systems. MDM system receives data and information from multiple external systems. Typical MDM technologies provide various mechanisms for handling real-time or near-real-time data synchronization with the external systems by tracking data changes during save process and propagating the data changes. This is conventionally done through scheduled jobs, Web services or such other mechanisms.
In the presence of multiple external systems, an MDM system which is not optimally designed to handle highly transactional and volatile data, starts to degrade in performance. Other challenges include limitation of flexibility in scaling an enterprise-wide infra-structure by adding more systems. Currently for each additional external system that is added to the enterprise-wide infrastructure, there needs to be web service or a scheduled job for communicating with the MDM system. MDM systems' communication with the external systems also is typically “same-data-to-all-systems” and this accentuates the performance problem for MDM system. Traditionally, data synchronization with an ‘Operational Data Store’ (ODS) does “full data” synchronization. This “full data” synchronization can be costly in terms of performance. For some external systems, ‘data marts’ are used to solve specific business problems. Data marts too use data extraction from MDM, by creating a database model for the business problem of interest. This database model can derive data from multiple sources in the MDM. However, data marts feed only ‘downstream’ systems and they typically do not derive data from a single source in the MDM. This limits their use to specific business problem they are designed for and also in terms of their flexibility.